Firearm and/or firearm sight calibration and/or zeroing

ABSTRACT

A method for calculation of firearm sight calibration instructions by processing an image of a shooting target. The method comprises receiving at least one image from at least one imaging device, the at least one image depicts a shooting target with a shot group comprising a plurality of bullet holes, receiving an indication of a firearm type and/or a firearm sight type to calibrate, using at least one hardware processor for calculating firearm sight calibration instructions according to at least one relative location of one of the plurality of bullet holes in relation to another of the plurality of bullet holes by processing the at least one image and at least one predefined parameter of the firearm and/or the firearm sight type, and outputting presentation instructions to present the firearm sight calibration instructions on a display of a client device.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/090,821 filed on Oct. 3, 2018, which is a National Phase of PCTPatent Application No. PCT/IL2018/050355 having International filingdate of Mar. 27, 2018, which claims the benefit of priority of IsraelPatent Application No. 251490 filed on Mar. 30, 2017. The contents ofthe above applications are all incorporated by reference as if fully setforth herein in their entirety.

FIELD AND BACKGROUND OF THE INVENTION

The present invention, in some embodiments thereof, relates tocalibration and, more specifically, but not exclusively, to automaticfirearm sight calibration, also known as firearm zeroing.

Over the years, various techniques and devices have been developed tohelp a person accurately aim a firearm, such as a rifle or targetpistol. One common approach is to mount on the firearm's barrel a sightor scope, through which the person views the intended target inassociation with a reticle, often with a degree of magnification.Although existing firearm sights have been generally adequate for theirintended purposes, they have not been satisfactory in all respects.

For example, when a sight is first mounted on the barrel and/or body ofa firearm, it needs to be aligned or “zeroed” with the firearm barrel,typically through a trial and error process. For example, a person mayshoot one or more bullets at a target which is a known distance away,identify the extent to which the bullets strike the target at locationsoffset from the location at which the person was aiming, and then adjustthe alignment of the sight in relation to the firearm in a mannerintended to eliminate the offset. This sequence of steps is repeated inan iterative manner, until bullets are striking the target atsubstantially the same location where the person is aiming.

This process results in alignment of the sight and firearm and needs ofa shooter for one specific set of conditions. However, during subsequentuse of the firearm and sight, for example when hunting, a variety ofconditions can vary from the conditions that existed during thealignment or zeroing process, and can thus affect the trajectory of abullet. These include factors such as temperature, pressure, humidity,wind speed and wind direction, all of which affect the density of airand thus the drag exerted on the bullet, and drag in turn influences thetrajectory. Further, the tilt of the firearm barrel can influence thedirection in which gravity acts on the bullet in relation to the initialtrajectory of the bullet, and this can in turn influence how gravity

Firearms such as rifles and handguns have adjustable sights. In M16A2Rifle for example elevation adjustments are made using the front sight,and elevation changes and windage adjustments are made using the rearsight.

In M16A2 Rifle the rear sight has an elevation knob with rangeindicators from 300 to 800 meters and two apertures for range. Oneaperture is marked 0-2 for short range from 0-200 meters and an unmarkedaperture for normal range from 300 to 800 meters. The 0-2(large)aperture is used for short range when the rear sight is all the waydown. The 8/3 (300-meter) mark on the elevation knob is aligned with theindex mark on the left side of the receiver. The rear sight alsoconsists of a windage knob on the rear side of the sight. Each desecrateturning action of the windage knob, known as a click, moves a strike ona round from ⅛ inch (0.3 centimeters) at 25 meters to 4 inches (10centimeters) at 800 meters. In M16A2 Rifle a windage scale is on therear of the sight and the windage knob pointer is on the windage knob.To adjust windage or move the strike of the round, the windage knob isturned counterclockwise to move the strike to the left and clockwise tomove the strike to the right. To adjust elevation the elevation knob isturned until the desired range is indexed at the index mark on the leftside on the sight.

In order to calibrate the rear sight a mechanical zero is usuallyestablished on the rifle and then a target such as 25 meters zeroingtarget (e.g. see FIG. 1A) is used on shooting sessions with the firearmto be calibrated. After each shooting session a shot group is manuallyidentified in the target, for example a shot group of three bulletholes. When the shot group is not within a defined region such as acircle on a silhouette of the target, squares on the target are used todetermine the required adjustment. The squares are usually numberedaround the edges of the target to equal the number of clicks required torelocate the shot group to the circle. The number of squares may bedetermined in relation to a center of the bullet holes which comprisethe shot group is identified. The center may be calculated by marking afirst line between bullet holes of one pair of bullet holes and thenmarking a second line between a third bullet hole and the center of thefirst line. This allows manually marking a center of the second line asthe center of a shot group that comprises 3 bullet holes.

Additional shooting sessions are continuously held to form a sequence ofshot groups and making corrections between the shooting sessions until ashot group is identified in the circle on the silhouette. Calibration isachieved when a shot group is within a target region such as the circle.Calibration is achieved when the shot group within the target region istight, for example holes are not located at a distance of more than fewcentimeters from, one another.

SUMMARY OF THE INVENTION

According to some embodiments of the present invention there is provideda method for calculation of firearm sight calibration instructions byprocessing an image of a shooting target. The method comprises receivingat least one image from at least one imaging device, the at least oneimage depicts a shooting target with a shot group comprising a pluralityof bullet holes, receiving an indication of a firearm sight type of afirearm to calibrate, using at least one hardware processor forcalculating firearm sight calibration instructions according to at leastone relative location of one of the plurality of bullet holes inrelation to another of the plurality of bullet holes by processing theat least one image and at least one predefined parameter of the firearmsight type, and outputting presentation instructions to present thefirearm sight calibration instructions on a display of a client device.

Optionally, wherein the indication in a firearm type.

Optionally, wherein the at least one imaging device and the processorare integrated into a common handheld device.

Optionally, the method comprises: storing the plurality of bullet holesas a plurality of historical bullet holes and receiving at least one newimage depicting the shooting target with a new shot group comprising aplurality of new bullet holes, the plurality of new bullet holes areidentified in a calculation excluding the plurality of historical bulletholes. The at least one hardware processor is used for: calculating aplurality of new distances between the plurality of new bullet holes byprocessing the at least one new image, calculating new firearm sightcalibration instructions according to the plurality of new distances andthe at least one predefined parameter and outputting new presentationinstructions to present the new firearm sight calibration instructionson the display.

Optionally, wherein the at least one hardware processor is used for:calculating a shot group analysis based on a distribution pattern of theplurality of bullet holes in relation to the shooting target. Thepresentation instructions comprise a recommendation selected accordingto an outcome of the shot group analysis.

Optionally, wherein the at least one hardware processor is used for:analyzing an output of at least one sensor for an identification of anending of a shooting event and triggering the calculation in response tothe identification.

More optionally, wherein the at least one sensor is a member selectedfrom a group consisting of: an audio sensor, an accelerometer, agyroscope sensor and an imaging sensor monitoring a front end of abarrel of the firearm.

More optionally, wherein the at least one sensor is integrated in amobile device used by the user.

Optionally, wherein the firearm sight calibration instructions comprisean indication of a number and a direction of desecrate turning actionsfor turning a sight calibration knob.

Optionally, the method comprises presenting the shooting target througha graphical user interface (GUI); the shot group is marked by a user whouses the GUI.

Optionally, the method comprises: receiving a firearm unique identifiervalue, using the firearm unique identifier value for extractinginformation comprising at least one of: historical shot group data andhistorical firearm sight calibration instructions data, and estimating afirearm functionally according to an analysis of the extractedinformation and the an output of an analysis of the shot group.

More optionally, the firearm unique identifier value is received from ananalysis of a machine readable code imaged in at least one image.

More optionally, the firearm unique identifier value is received from agraphical user interface presented on a display of a mobile device.

Optionally, the method comprises: acquiring information comprising atleast one of: historical shot group data and historical firearm sightcalibration instructions data of a plurality of firearms of the firearmtype and calculating a firearm sight type functionality score accordingto an analysis of the acquired information and the output of an analysisof the shot group.

Optionally, the method comprises: acquiring historical data comprisingat least one historical shot group of the firearm or of at least oneother firearm of the firearm type and estimating a malfunction in thefirearm according to a match between the historical data and the shotgroup.

Optionally, the method comprises classifying the shot group according toa classifier created based on an analysis of a plurality of historicalshot groups of a plurality of other firearms or a plurality of firearmsights and/or manufacturer/professional instructions.

According to some embodiments of the present invention there is provideda handheld device of calculation firearm sight calibration instructionsby processing an image of a shooting target. The handheld devicecomprises at least one image sensor adapted to capture at least oneimage, the at least one image depicts a shooting target with a shotgroup comprising a plurality of bullet holes, an interface adapted toreceive an indication of a firearm sight type of a firearm to calibrate,a memory storing a code, at least one hardware processor adapted toexecute a code for calculating firearm sight calibration instructionsaccording to at least one relative location of one of the plurality ofbullet holes in relation to another of the plurality of bullet holes byprocessing the at least one image and at least one predefined parameterof the firearm type, and a presentation unit adapted to present thefirearm sight calibration instructions.

Unless otherwise defined, all technical and/or scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the invention pertains. Although methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of embodiments of the invention, exemplarymethods and/or materials are described below. In case of conflict, thepatent specification, including definitions, will control. In addition,the materials, methods, and examples are illustrative only and are notintended to be necessarily limiting.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Some embodiments of the invention are herein described, by way ofexample only, with reference to the accompanying drawings. With specificreference now to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

In the drawings:

FIGS. 1A-1E are images of commonly used shooting targets from a numberof points of view;

FIG. 2A is a schematic illustration of a calibration system foranalyzing one or more images of a shooting target during a shootingsession to calculate firearm sight calibration instructions, accordingto some embodiments of the present invention;

FIGS. 2B-2E are images of commonly used shooting targets from a numberof points of view;

FIG. 3A is a flowchart of an exemplary process for calculatinginstructions for calibrating a firearm based on an analysis of one ormore images of a shooting target, according to some embodiments of thepresent invention;

FIG. 3B is a flowchart of an exemplary process for assuring quality oftarget images, according to some embodiments of the present invention;

FIG. 4 is a commonly used shooting target having a shot group with aplurality of bullet holes;

FIG. 5 is a process for identification of bullet holes assisted by amatching process wherein image(s) depicting the shot group, according tosome embodiments of the present invention;

FIG. 6 is a screenshot of a window comprising textual and graphicalinstructions indicative of required actions, according to someembodiments of the present invention; and

FIGS. 7A-7C are respectively screenshots of windows of a GUI presentinghole clusters captured during a calibration session, according to someembodiments of the present invention.

DETAILED DESCRIPTION

The present invention, in some embodiments thereof, relates tocalibration and, more specifically, but not exclusively, to automaticfirearm sight calibration analysis and calculation.

According to some embodiments of the present invention there areprovided methods, devices and systems of generating instructions forcalibrating (zeroing) a sight of firearm by processing one or moreimages of a shooting target to detect relative location of bullet holesof a shot group in relation to a reference area (e.g. center of target)and automatically deducing therefrom calibration instructions. Thecalibration instructions are optionally deduced for a firearm (e.g.M-16, Tavor, M-4 etc.) or a firearm sight (e.g. telescopic, Red-Dot, oriron sights) and/or a combination thereof. Optionally, the calibrationinstructions are calculated and presented on hardware of a handhelddevice such as a mobile phone or a tablet. Optionally, the calibrationinstructions are adapted to the firearm and/or firearm sight type. Theshot group includes bullet holes formed by bullets shot from a firearmto be calibrated in a single shooting session. A shooting session is oneout of one or more sessions of a calibration process. Optionally,historical shot groups which are identified in a certain shooting targetare recorded and removed from a process for detection of a new shotgroup.

According to some embodiments of the present invention there areprovided methods, devices and systems estimating functionality levelsand/or malfunctions of a specific firearm, firearms of a certain firearmtype, and/or sights based on an analysis of a dataset of a plurality ofshot groups and/or plurality of calibration recommendations.

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings and/or the Examples. The invention iscapable of other embodiments or of being practiced or carried out invarious ways.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network.

The computer readable program instructions may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider). In some embodiments, electronic circuitry including, forexample, programmable logic circuitry, field-programmable gate arrays(FPGA), or programmable logic arrays (PLA) may execute the computerreadable program instructions by utilizing state information of thecomputer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Reference is now made to FIG. 2A which is a schematic illustration of acalibration system 100 for analyzing one or more images of a shootingtarget 116, zeroing target, during shooting session(s) to calculatefirearm sight calibration instructions based on an analysis of an imagedepicting the distribution of shooting holes of a shot group in theshooting target 116, according to some embodiments of the presentinvention. The firearm sight calibration may be a sight of any firearm,for instance automatic, semi-automatic (self-loading) and manualfirearms and/or a combination of a firearm sight and a firearm type. Thefirearm may be a rifle or a gun.

As used herein, a shot group is a group of bullet holes formed in ashooting target 116 by bullets shot by a firearm targeted forcalibration at a single shooting session for example 5 rounds ofshootings at the shooting target 116 when it is placed at a 25 meterdistance or 100 meter distance from the shooter. As used herein, afirearm type, a firearm sight type or a combination thereof may bereferred to herein interchangeably.

Calibration instructions are instructions to a shooter 115 to manuallycalibrate a firearm, for instance by rotating a blade or a knob and/orto operate a mechanical mechanism for adjusting a sight location.Calibration instructions may be referred to as zeroing instructions. Thecalibration system 100 includes a processor 101 executing a calibrationcode 102 for calculating firearm sight calibration instructions, amemory 103 hosting the calibration code 102, one or more image sensor(s)104 such as camera(s) for capturing one or more images of the shootingtarget 116 and optionally one or more sensors 105 such as accelerometerswhich monitor usage of a firearm by a user 115 during the shootingsession. Optionally, the image sensor is camera enhanced with atelescopic optical arrangement to allow acquiring an image of theshooting target from a range of 25 meter (m). The calibration code 102may be an app store application or an application installed directly ona client device. The calibration code may be program instructions storedin a non-transitory medium, for example a software application executedon one or more processing nodes having one or more processors. Thecalibration system 100 may be or include a handheld device, such as amobile phone or a tablet or a smartwatch. The processor of thecalibration system 100 may reside on, a server, a computer, a desktop, atablet, a smartphone which communicates with the handheld device. Theimage sensor(s) 104 may be, for example, an image sensor such as acamera, a stereoscopic camera, a night-vision camera and/or an infra-red(IR) camera which are optionally integrated into the handheld device.The sensors 105 may include, for example, an audio sensor such as amicrophone, an accelerometer, a gyroscope and an imaging sensor whichoptionally monitor a front end of a barrel of the firearm. Thecalibration code may include instructions to operate a user interface,for example, a graphical user interface (GUI) designed to be presentedto user 115 on a presentation unit such as a speaker or a display 107,optionally the integrated speaker or display of the handheld device. TheGUI maybe used to allow the user input information such as firearmand/or sight type, model and/or unique identifier (ID).

Optionally, the system 100 connects to one or more remote services 110,for example, a database and/or an analytics service for generatingreports and/or for storing outputs of the calibration process and/orshot group data. The remote services 110 may be available through, forexample, a cloud service, a module executed on one or more remoteprocessing nodes, for example, a server and the likes.

Reference is also made to FIG. 3A which is a flowchart of an exemplaryprocess for calculating instructions for calibrating a firearm based onan analysis of one or more images of a shooting target, according tosome embodiments of the present invention. The process 300 may be usedfor calculating instructions for calibrating a firearm such as a gun, apistol and/or a rifle or sight. The process 300 may be executed on aprocessor(s) of a handheld device such as a Smartphone or a tablet orone or more processing nodes, for example, a local processing node, suchas a computer, a server and/or a cluster of processing nodes and/or on aremote processing node, for example, a remote server, a remoteserver-cluster and/or a cloud service. Optionally, the process 300 issplit between two or more of the processing nodes such that each of theprocessing nodes executes one or more segments of the process 300.

As shown at 301, the process 300 starts with receiving one or moreimages from one or more image sensor(s) such as the imaging device 104.The one or more images depict the shooting target 116 with a shot groupof a plurality of bullet holes formed by bullets during a shootingsession conducted by the user 115. A shot group may include 3, 5, 7 orany intermediate or larger number of bullet holes.

Optionally, a welcome or initialization window is presented to the userwhen initializing a respective application. The welcome orinitialization window may include instructions of how to capture theimages from one or more image sensor(s) such as the imaging device 104or how to upload images from a repository. The welcome or initializationwindow may also gather other information such as firearm ID, user ID,number of calibration sessions, age, gender, experience level and/or thelike. The information may be used for learning as further describedbelow.

The one or more images may be for example video frames or images whichare captured using an image sensor of the handheld device. The one ormore images may be acquired in response to press of a button in the GUIand/or automatically when the shooting target is identified by imageprocessing in frames of a video captured by the image sensor of thehandheld device. The acquisition is optionally held in real time, forinstance less than a second after shooting target is identified by imageprocessing in frames.

Optionally, the one or more images are images having an image qualityabove a predefined threshold. The image quality is optionally aperceived image degradation measure for instance compared to an ideal orperfect image. The measure may be based on the presence or absence ofdistortion or artifacts in the image. The measure may be based onSharpness, Noise, Dynamic range, Tone reproduction, Contrast, Coloraccuracy, Distortion, Vignetting, Exposure, Lateral chromatic aberration(LCA), Lens flare, and/or Artifacts.

Optionally, quality of the received images is estimated, for example byexecuting a quality estimation function. For instance, when the qualityis below a threshold received images are ignored and when the quality isabove the threshold the image is used. In such a scheme, only imageswith sufficient sharpness and/or contrast are analyzed. Optionally,image metadata may also be extracted, for instance Other imageacquisition conditions may verified, for instance a target centeralignment which indicates when the target center is aligned with thecenter of the Field Of View (FOV) (e.g. determined by alignmentparameters), FOV coverage that indicates when the FOV covers the entiretarget and optionally not too much area around the target, axisalignment that indicates that the image axes are sufficiently alignedwith the target axes (no excessive in-plane rotation), perspectivedistortion level and/or the like. The image quality analysis may alsoindicate and/or used for alerting the user about unsuitable targets.This is as conditions may not be not met when the target is excessivelydirty, partially torn, contains an excessive amount of bullet holesand/or the like.

Now, as shown at 302, an indication of a firearm sight type of a firearmto calibrate is received. The indication may be a user selectionidentified using the GUI and/or automatically by analyzing thecharacteristics of the bullet holes, for instance diameter and/or shapeand/or fire noise as recorded by the sensor(s) 105. The indicationallows selecting and/or calculating parameters for bullet hole detectionand calibration instructions for example as described below. Theparameters may be an outcome of a machine learning process and/or basedon desired correlation as exemplified below. Optionally, the firearmsight type of the firearm is set as a default and not manually changed.Firearm sight type and firearm type are referred to hereininterchangeably as native firearm sight type which may be automaticallydeduced from firearm type indication. Optionally, a user provides bothfirearm sight type and firearm type so as to allow deducing calibrationinstructions based on parameters of the firearm sight and firearm typeitself (e.g. caliber, barrel length etc.).

As shown at 303, bullet holes of the shot group are identified based ona mathematical model and/or morphology. The mathematical model of bulletholes may be defined as connected components which are darker comparedto surrounding pixels and having an absolute threshold and a relativethreshold (difference), an area of bullet hole region, and/or ageometrical shape of bullet holes of a firearm, optionally of theindicated firearm type. Optionally, a firearm sight type record isacquired in response to the user selection 302 and used for extractingparameters for the mathematical model. Optionally holes are detected asdescribed in International Journal of Applied Mathematics Volume 27 No.6 2014, 549-566 ISSN: 1311-1728 which is incorporated herein byreference. Optionally, coordinates of each bullet hole are stored in ashot group record.

Bullet holes and/or distribution thereof may be identified based onvisual descriptors using machine learning techniques. In suchembodiments visual descriptors (also known as visual words) such asScale Invariant Feature Transform (SIFT) local feature descriptor areused to represent bullet holes and/or shot groups in multiple images tofacilitate supervised or unsupervised learning of a classifier forbullet holes and/or shot group detection and classification, see DavidG. Lowe, Distinctive Image Features from Scale-Invariant Keypoints,Computer Science Department University of British Columbia Vancouver andVideos Josef Sivic and Andrew Zisserman, Google: Efficient Visual Searchof Videos, Department of Engineering Science University of Oxford whichare incorporated herein by reference. The multiple images for supervisedor unsupervised learning may be images gathered by the system 210 forthe generation and presentation of calibration instructions as describedherein.

Now, as shown at 304, a calculation of relative location of the shotgroup in relation to a reference area of the shooting target, forinstance the center of a shooting target is made. The shot grouplocation may be defined as a center calculated based on the location ofthe bullet holes. The shot group center is calculated by processing theone or more images using the processor(s) 101 to identify the locationof the bullet holes in relation to one another, for instance using thedata from the respective shot group record. For example, FIG. 4 is anexemplary shooting target 401 having a shot group 404 with a pluralityof bullet holes such as 402. FIG. 4 depicts how the center isautomatically calculated. First, a first line (distance) 405 isidentified between holes of one pair of holes. Then, a second line(distance) 406 is identified between the third hole and the center ofthe first line. Now, a center of the second line 403 is identified asthe center.

In another example, a line is automatically calculated between bulletholes of each of two pairs of bullet holes which are most proximate toone another. Then, an additional line is automatically calculatedbetween centers of the lines. Then, a further additional line isautomatically calculated to the center of the additional line and anunpaired drill hole. This allows dividing the further additional line tofive and identifying the fourth line away from the unpaired drill holeas the center of a shot group that comprises 5 bullet holes.

Now, as shown at 305, firearm sight calibration instructions areautomatically calculated or selected according to the relative locationof the shot group and/or a geometric feature of the shot group, such asa geometric feature and the center of mass, in relation to a referencearea of the shooting target and one or more predefined parameters of thefirearm type. For instance, a plurality of firearm sight type recordsare used. Each firearm sight type record includes a conversion datasetor functions for converting a relative location of the shot group inrelation to a reference area of the shooting target into firearmspecific calibration instructions. For instance, calibrationinstructions may be turning a sight knob such as a windage knob or anelevation knob according to a number of turns which is determined basedon the distance and/or direction from the reference area. Otherinstructions may be to rotate a sight blade and/or windage screw with azeroing wrench, see for example operator manual of TAVOR™ SAR.

This allows, as shown at 306, outputting presentation instructions topresent the firearm sight calibration instructions on the display 107.The calibration instructions may be textual and/or graphicalinstructions indicative of user actions for calibration of the firearm,for example indicative of a knob or a blade to turn, a number of adesecrate turning actions such as knob/blade turns, and a knob/bladeturning direction. Exemplary calibration instructions can be textual,for example, “turn the knob 3 clicks to the left”, “turn the blade 2clicks up” and or graphical, for instance an animation or an imageindicating direction and number of clicks for adjustment. For example,see FIG. 6 which is a screenshot of a window comprising textual andgraphical instructions indicative of a required actions and the locationof the knobs for performing the required actions. The instructions areoptionally color coded. Alternatively or additionally, the firearm sightcalibration instructions are sent to a manipulation mechanism thatadjusts the sights accordingly, for example a mechanical arm that iscontrolled by a controller that receives the firearm sight calibrationinstructions and translate them into mechanical sight adjustmentactions.

The calibration instructions may be determined based on a classificationdetermined by applying machine learning processes on historical datagathered using the system, for instance improvements between twoconsecutive shot groups of two consecutive shooting sessions.

As shown at 307, the process of calculating and outputting forpresentation of calibration instructions may be iteratively repeatedwherein in each iteration a new shot group is identified to calculatenew calibration instructions based on the assumption that the previouscalibration instructions have been implemented by the user 115 and/or amanipulation mechanism. Optionally, shot groups which are capturediteratively are associated or otherwise connected as a sequence. Thissequence may be analyzed to identify a pattern indicative of acalibration and/or firearm and/or sight firearm malfunction and/or thelike.

According to some embodiments of the present invention, an analysis ofsignals obtained from one or more sensors of the calibration system 100,for example a mobile device, is performed for assuring quality of targetimages. This may be required as a distance, position and/or orientationof the target in relation of the calibration system 100 is not fixed.The process for assuring quality of target images is optionally based onone or more images captured by the image sensor(s) 104. The analysis canidentify and specify the reason for erroneous target capture, at thetime of capture or afterwards. Such identification can prevent furtheranalysis and/or provide corrective instructions to the user.

The analysis may occur in real-time or near real-time, either as apreprocessing step, or interactively, and/or in each shooting sessionand/or for assuring that necessary conditions for image capturing aremet.

FIG. 1B is an image of a calibration target, FIG. 1C is an image of acorrect capture of the calibration target and FIGS. 1D and 1E are imagesof several less favorite situations detected by reading one or moredevice sensors.

FIGS. 2B-2E depict several frames taken from less favorite points ofview. FIG. 2B is an image of a perspective distortion induced bycapturing the target image from a side position, FIG. 2C is an imagecaptured from a distance and FIGS. 2D-2E are respectively an imagecaptured from a great proximity such that no lateral borders arecaptured and using a zoom-in function. When the captured frames arecaptured from such points of view accuracy of measurement maybe reduced.When only a part of the calibration target is visible, drills can bemissed simply because they are outside the captured frames. Furthermore,for example as depicted in FIG. 2D, frames captured from close proximityare blurred, which also may affect the ability to properly detect andlocate hits. Measurement can trivially fail due to a false capture—theuser erroneously or in purpose capturing another object.

FIG. 3B is a flowchart of a process for exemplary process for assuringquality of target images, according to some embodiments of the presentinvention. The process for assuring quality of target images may beperformed during or before 301 which is described above. For example,the process may be performed before the shooting session. In theseembodiments, as shown at 910, analysis begins by reading the orientationand acceleration values. When the device is rotated, tilted oraccelerating, an alert may be presented to the use together with anexplanation and/or suggested correction, and further image analysis isinhibited.

Proceeding with analysis, according to some embodiments, AS shown at920, a template target may be used, for example selected from a numberof templates in different possible scales (920). A bulls' eye patternmay be used as a template, see for example Fast Multi-scale TemplateMatching Using Binary Features, Feng Tang and Hai Tao, IEEE Workshop onApplications of Computer Vision (WACV'07) which is incorporated hereinby reference.

As shown at 930, failure to locate the template (930) may induce apresentation to the user such as “no target is found”.

As shown at 940, computed template scale and position are used toindicate a number of capturing errors, for example excessive distance,zoom-out, excessive zoom-in and/or impossible cropping. Again, an alertmay be produced together with an explanation and/or suggested correctionwhich is deduced from an image analysis, for example “move closer”,“move away”, and “keep entire target in the field of view”. A scale thatproduces a maximum matching score and position of that score provide acoarse estimate of the image transform, mapping a known image pattern,to the captured image. As shown in FIG. 1A, a calibration target willcomprise a grid made of 2 perpendicular sets of parallel lines. Apossible approach to estimating perspective from set of parallel linesis based on computing the vanishing point; see for example [Yin,Xu-Cheng et al. (2011). Robust Vanishing Point Detection forMobileCam-Based Documents. Proceedings of the International Conferenceon Document Analysis and Recognition, ICDAR. 136-140] which isincorporated herein by reference.

Extreme perspective distortion may occur whenever optical axis of thecamera is not perpendicular to the target plane. After the estimation ofperspective (950) an alert is produced together with anexplanation/suggested correction (“move to the center of the target”),and further image analysis is inhibited.

As shown at 980 mf 970, image quality is determined based on analysis. Apossible measure of quality may be image blur. Another measure ofquality may be a fixed disturbance (e.g. dirt on the camera lens). Onlywhen the minimum image quality is verified image analysis for hitdetection, center and spread analysis may proceed. Although the analysisin FIG. 3B is described in a post-capture mode, a similar or partialflow of actions may be executed in real-time so that the user can reactto the “capture now” or corrective action suggestions thus acceleratingthe calibration process.

According to some embodiments of the present invention, bullet holeswhich are recorded as identified to one shooting session are ignored inthe following shooting session. This is optionally facilitated byrecording a shot group and ignoring the respective bullet holes whenidentifying a new shot group. In such embodiments, the bullet holes of ashot group are stored as historical bullet holes and when new image(s)depicting a shooting target with a new shot group is received new bulletholes are identified. The new bullet holes are identified in acalculation excluding the historical bullet holes. This process may beiteratively repeated to allow calculating a plurality of calibrationinstructions, either to the same firearm or for different firearmsand/or sights of the same user or of difference users.

For example, reference is made to FIG. 5 wherein identification ofbullet holes is further assisted by a matching process wherein image(s)depicting the shot group are compared with previously captured image(s)of the target. In such a manner, bullet-like regions that appeared inthe previous image (set) are excluded from analysis in the new image(s).This would prevent previous holes, hole-like target markings (such asnumerals) and hole-like spots to be considered.

Optionally, as described in FIG. 5, the matching process comprises of ageometric alignment process wherein the new images are brought intogeometric registration with the previously captured image(s)prior tocomparison, for instance by subtraction. The process described in FIG. 5may be part of 304.

As shown at 501, the new images and the previously captured images arealigned by a coarse image alignment that includes keypoint detection,descriptor calculation, and keypoint matching. Then, as shown at 502, ahomography transformation between the images is solved. Optionally,newly added bullet holes are ignored when executing the homographytransformation between the images, for instance by adopting a schemerobust to outliers, in which the newly added bullet holes are rejectedas outliers. Such a scheme is provided by RANSAC, see also [Randomsample consensus: a paradigm for model fitting with applications toimage analysis and automated cartography, Fischler and Bolles, Comm.ACM, Volume 24 Issue 6, June 1981, Pages 381-395.

The coarse alignment step may be alternatively performed according to areference to a location of a known structure of the shooting target. Forexample, FIG. 1A depicts a target which customarily includes a“bullseye” pattern in the center of the target and an array ofhorizontal and vertical grid lines which are used for human measurementof bullet holes positions. These elements may be used as a reference.For instance, the bullseye pattern may be used to align the centers ofthe new image(s) with the center the previous image(s), optionally whenthe bullseye pattern and/or the grid lines pattern (or both) providefurther scaling and/or rotation parameters, bring the new and theprevious image into correspondence.

While the coarse alignment process is usually sufficient to excludeprevious holes, it may be advantageous to perform further fine-alignmentbetween the new image(s) and the previous image(s) (set). Such a processmay be done as described in B. D. Lucas and T. Kanade (1981), Aniterative image registration technique with an application to stereovision. Proceedings of Imaging Understanding Workshop, pages 121-130.

Now, as shown at 504, image change is detected by comparing between thenew images and the previously captured images. This allows identifyingthe new holes. This may be done using image subtraction, for example asdescribed in image change detection algorithms: a systematic survey, R.J. Radke et al, IEEE Transactions on Image Processing, 14 (3) March2005. For example, newly added holes are detected in a difference imagecalculating based on the differences between the new images and thepreviously captured images. This allows detecting a new holeintersecting with a previous hole, that otherwise may be detected as thesame hole. An additional benefit of image alignment and subtraction isthat it is sufficient to limit detection of holes to the differencezones, thereby greatly reducing the computing resources especially whenprocessing is performed on a mobile device.

FIGS. 7A-7C are respectively screenshots of windows of a GUI presentinghole clusters captured during a calibration session. In each of thefirst two screenshots marks, colored dots, are added to holes of onecluster out of two clusters detected during the calibration session. Inthe third screenshots both clusters are marked and an indication ofweapon calibration is presented. Optionally, holes of one cluster areseparated from holes detected before the holes of the cluster areformed. For example, see how in FIG. 7C different clusters are coloreddifferently (e.g. green and red—might not be shown in black and whiteprinting).

Prior to image subtraction it may be necessary to correct forillumination differences, for example by a known process of histogrammatching, so that pixel difference shall be affected by actual changes(such as new bullet holes) and not by illumination changes.

According to some embodiments of the present invention, a distributionpattern of the bullet holes of the shot group is calculated in relationto the shooting target. The distribution pattern may be indicative of aninaccurate shot group that is an outcome of unsteady shooting andtherefore cannot be used for calibration. In such embodiments, the GUImay be instructed to present a recommendation to conduct an additionalshooting session. The recommendation may detail an estimated reason forthe unsteady shooting. For example, when the distribution patternincludes a plurality of holes which are arranged along a vertical line,the recommendation is to exhale before shooting and when thedistribution pattern includes a plurality of holes which are arrangedalong a horizontal line, the recommendation is to perform shooting witha firmer grip.

According to some embodiments of the present invention, a shot group isidentified after a shooting session ending is automatically detected.The ending may be determined by an analysis of an output of thesensor(s), for instance after 3, 5, or 7 or any intermediate or largernumber shoots are audibly identified and/or when 3, 5, 7, or anyintermediate or larger number of new bullet holes are detected byiteratively analyzing images of the shooting target. In suchembodiments, an image sensor is mounted to continuously image theshooting target. For example when the system is implemented as ahandheld device, the handheld device may be mounted on a holder thatdirects the image sensor of the handheld device toward the shootingtarget. The identification of the shot group triggers 303-306,optionally iteratively.

According to some embodiments of the present invention, the calibrationrecommendation and/or data documenting the shot group are stored inassociation with a unique ID value of the firearm. The unique ID may bemanually inputted using the GUI and/or automatically identified, forinstance by reading a machine readable code such as a tag or a barcodeassociated with the firearm or the shooting target. For example, a QRcode may be read from the shooting target to indicate the unique IDvalue of the firearm. In another example, the machine readable code is aQR barcode that is read in advance from firearm documentation. In suchembodiments, firearm performances and/or calibration state can betracked. Optionally, a user and/or an administrator are alerted by thesystem when a firearm was not calibrated for more than a defined periodand/or after a known usage. Optionally, the shot group data and/orcalibration data are used for calculating a firearm trend. The trend mayindicate when the firearm should be sent for probing and/or for factoryresetting.

According to some embodiments of the present invention, the calibrationrecommendation and/or data documenting the shot group are combined withcalibration recommendation and/or data documenting historical shotgroups of the specific firearm or of firearm and/or sight of a commontype. This allows identifying trends of a firearm sight typefunctionally and/or evaluating firearm sight calibration drift orperformances for a period.

According to some embodiments of the present invention, shot groupswhich are acquired in iterative shooting sessions of a calibrationprocess of a firearm and/or sight are stored and tagged, for examplewith time stamp, a firearm type, a firearm ID, a shooter ID and/or thelike. This allows creating a dataset that documents sequences of shotgroups which are held in different calibration processes, for example byfirearms of the same type and/or of the same firearm over time. Suchdataset may be analyzed to identify trends and/or patterns of a certainfirearm or of firearms of the same type. Such dataset may be used tocreate a classifier for classifying a current sequence of shot groups,for example to determine a malfunction in the firearm or the sightfirearm. The trends and/or patterns may be indicative of a functionalitylevel of firearms of the same type, for example a change in the neededfrequency of firearm calibration over time and/or in correlation with anumber of calibration sessions, a time related change in the accuracy offirearms of the same type, reaction of firearms of the same type to thetime of the day and/or reaction of firearms of the same type to the typeof the range (e.g. night range, day range, 25 m range, 100 m rangeand/or the like). The dataset may be used to set a firearm baseline.This allows using the firearm baseline to identify malfunctioningfirearms and/or to assist in evaluating the performances of shooter, forinstance by using the baseline for normalizing the effect of firearmaging and/or over usage.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

It is expected that during the life of a patent maturing from thisapplication many relevant systems and methods will be developed and thescope of the term a processor, an imaging device, and a sensor isintended to include all such new technologies a priori.

As used herein the term “about” refers to ±10%.

The terms “comprises”, “comprising”, “includes”, “including”, “having”and their conjugates mean “including but not limited to”. This termencompasses the terms “consisting of” and “consisting essentially of”.

The phrase “consisting essentially of” means that the composition ormethod may include additional ingredients and/or steps, but only if theadditional ingredients and/or steps do not materially alter the basicand novel characteristics of the claimed composition or method.

As used herein, the singular form “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise. For example,the term “a compound” or “at least one compound” may include a pluralityof compounds, including mixtures thereof.

The word “exemplary” is used herein to mean “serving as an example,instance or illustration”. Any embodiment described as “exemplary” isnot necessarily to be construed as preferred or advantageous over otherembodiments and/or to exclude the incorporation of features from otherembodiments.

The word “optionally” is used herein to mean “is provided in someembodiments and not provided in other embodiments”. Any particularembodiment of the invention may include a plurality of “optional”features unless such features conflict.

Throughout this application, various embodiments of this invention maybe presented in a range format. It should be understood that thedescription in range format is merely for convenience and brevity andshould not be construed as an inflexible limitation on the scope of theinvention. Accordingly, the description of a range should be consideredto have specifically disclosed all the possible subranges as well asindividual numerical values within that range. For example, descriptionof a range such as from 1 to 6 should be considered to have specificallydisclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numberswithin that range, for example, 1, 2, 3, 4, 5, and 6. This appliesregardless of the breadth of the range.

Whenever a numerical range is indicated herein, it is meant to includeany cited numeral (fractional or integral) within the indicated range.The phrases “ranging/ranges between” a first indicate number and asecond indicate number and “ranging/ranges from” a first indicate number“to” a second indicate number are used herein interchangeably and aremeant to include the first and second indicated numbers and all thefractional and integral numerals therebetween.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable subcombination or as suitable in any other describedembodiment of the invention. Certain features described in the contextof various embodiments are not to be considered essential features ofthose embodiments, unless the embodiment is inoperative without thoseelements.

All publications, patents and patent applications mentioned in thisspecification are herein incorporated in their entirety by referenceinto the specification, to the same extent as if each individualpublication, patent or patent application was specifically andindividually indicated to be incorporated herein by reference. Inaddition, citation or identification of any reference in thisapplication shall not be construed as an admission that such referenceis available as prior art to the present invention. To the extent thatsection headings are used, they should not be construed as necessarilylimiting. In addition, any priority document(s) of this applicationis/are hereby incorporated herein by reference in its/their entirety.

What is claimed is:
 1. A method for calculation of firearm sightcalibration instructions based on processing one or more images of ashooting target, comprising: receiving at least one image from at leastone imaging device, the at least one image depicts a shooting targetwith a plurality of bullet holes; using at least one hardware processorfor calculating firearm sight calibration instructions for a firearmsight according to at least one relative location of one or more of theplurality of bullet holes in relation to one or more other bullet holesof the plurality of bullet holes by processing the at least one image;and outputting instructions to at least one of: a display of a clientdevice that presents the firearm sight calibration instructions and amanipulation mechanism that adjusts automatically the firearm sightaccording to the firearm sight calibration instructions.
 2. The methodof claim 1, further comprising identifying the plurality of bullet holesby: aligning the at least one image, calculating a difference imagebased on differences between the at least one image and at least onepreviously captured image of the shooting target, and separating betweenthe plurality of bullet holes and a plurality of historical bullet holesaccording to the difference image.
 3. The method of claim 1, wherein thefirearm sight calibration instructions are calculated according anindication of a firearm type of the firearm sight.
 4. The method ofclaim 1, wherein the instructions are to instruct the manipulationmechanism to adjust automatically the firearm sight according to thefirearm sight calibration instructions.
 5. The method of claim 1,wherein the at least one imaging device and the processor are integratedinto a common handheld device.
 6. The method of claim 1, furthercomprising: storing the plurality of bullet holes as a plurality ofhistorical bullet holes; when at least one new image depicting theshooting target with a new shot group comprising a plurality of newbullet holes is received: the plurality of new bullet holes areidentified in a calculation excluding the plurality of historical bulletholes, and the at least one hardware processor is used for: calculatinga plurality of new distances between the plurality of new bullet holesby processing the at least one new image; calculating new firearm sightcalibration instructions according to the plurality of new distances;outputting new presentation instructions to present the new firearmsight calibration instructions on the display when calibration is notachieved.
 7. The method of claim 1, wherein the at least one hardwareprocessor is used for: calculating a shot group analysis based on adistribution pattern of the plurality of bullet holes in relation to theshooting target; wherein the presentation instructions comprise arecommendation selected according to an outcome of the shot groupanalysis.
 8. The method of claim 1, wherein the at least one hardwareprocessor is used for: analyzing an output of at least one sensor for anidentification of an ending of a shooting event; triggering thecalculation in response to the identification.
 9. The method of claim 8,wherein the at least one sensor is a member selected from a groupconsisting of: an audio sensor, an accelerometer, a gyroscope sensor andan imaging sensor monitoring a front end of a barrel of the firearm. 10.The method of claim 8, wherein the at least one sensor is integrated ina mobile device used by the user.
 11. The method of claim 1, wherein thefirearm sight calibration instructions comprises an indication of anumber and a direction of desecrate turning actions for turning a sightcalibration knob.
 12. The method of claim 1, further comprisingpresenting the shooting target through a graphical user interface (GUI),the shot group is marked by a user who uses the GUI.
 13. The method ofclaim 1, further comprising: receiving a firearm unique identifiervalue; using the firearm unique identifier value for extractinginformation comprising at least one of: historical shot group data andhistorical firearm sight calibration instructions data; estimating afirearm functionally according to an analysis of the extractedinformation and the output of an analysis of the shot group.
 14. Themethod of claim 13, wherein the firearm unique identifier value isreceived from an analysis of a machine readable code imaged in at leastone image.
 15. The method of claim 13, wherein the firearm uniqueidentifier value is received from a graphical user interface presentedon a display of a mobile device.
 16. The method of claim 1, furthercomprising: acquiring information comprising at least one of: historicalshot group data and historical firearm sight calibration instructionsdata of a plurality of firearms of the firearm type; calculating afirearm sight type functionality score according to an analysis of theacquired information and the output of an analysis of the shot group.17. The method of claim 1, further comprising: acquiring historical datacomprising at least one historical shot group of the firearm or of atleast one other firearm of the firearm type; and estimating amalfunction in the firearm according to a match between the historicaldata and the shot group.
 18. The method of claim 1, further comprisingclassifying the shot group according to a classifier created based on ananalysis of a plurality of historical shot groups of a plurality ofother firearms or a plurality of firearm sights and/ormanufacturer/professional instructions.
 19. An imaging system forcalculation firearm sight calibration instructions by processing animage of a shooting target, comprising: at least one image sensoradapted to capture at least one image, the at least one image depicts ashooting target with a plurality of bullet holes; a memory storing acode; at least one hardware processor adapted to execute a code forcalculating firearm sight calibration instructions for a firearm sightaccording to at least one relative location of one or more of theplurality of bullet holes in relation to one or more other bullet holesof the plurality of bullet holes by processing the at least one image;and outputting instructions to at least one of: a display that presentsthe firearm sight calibration instructions and a manipulation mechanismthat automatically adjusts the firearm sight according to the firearmsight calibration instructions.
 20. A computer program product forcalculation of firearm sight calibration instructions by processing animage of a shooting target, the computer program product comprising anon-transitory computer readable storage medium having programinstructions embodied therewith, the program instructions executable bya processor to cause one or more computing units to: receive at leastone image from at least one imaging device, the at least one imagedepicts a shooting target with a plurality of bullet holes; calculatingfirearm sight calibration instructions for a firearm sight according toat least one relative location of one or more of the plurality of bulletholes in relation to one or more other bullet holes of the plurality ofbullet holes by processing the at least one image; and outputtinginstructions to at least one of: a display of a client device thatpresents the firearm sight calibration instructions and a manipulationmechanism that automatically adjusts the firearm sight according to thefirearm sight calibration instructions.